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INTRODUCTION 


A  series  of  experiments  at  the  Air  Force  Academy  was  designed  to  study 
characteristics  of  human  information  processing  in  vision  and  attention  during 
simulated  flight.  Some  of  these  studies  were  aimed  at  improving  pilot  per¬ 
formance  by  means  of  alternate  flight  control  monitoring  devices  such  as  displays 
placed  in  the  peripheral  fields  of  ambient  vision.  Other  studies  were  aimed  at 
investigating  the  training  applications  of  these  alternate  forms  of  displays 
(e.g.,  Bermudez,  Schwank,  Longridge,  Smith,  and  McCloy,  1979;  Schwank, 

Bermudez,  Smith,  and  Harris,  1979). 

The  present  study  is  related  to  both  of  these  research  aims.  In  both 
lines  of  research,  both  system  and  operator  performance  data  was  obtained  under 
conditions  that  approximate  relevant  information  processing  demands  and  realistic 
motor  task  demands.  For  example,  the  previous  studies  have  required  subjects  to 
perform  mental  activity  during  either  the  difficult  vertical  S-"A"  aerial  maneu¬ 
ver,  or  during  an  instrument  landing  approach.  The  studies  employed  probed 
memory  techniques,  modified  after  the  Sternberg  technique  (1969),  to  Impose 
stress  while  the  pilot  flew  the  simulator.  Although  these  earlier  studies  have 
shown  memory  probes  are  very  demanding,  the  type  of  probe  used  seemed  much  less 
difficult  than  the  types  of  processing  demands  and  probes  that  typically  con¬ 
front  the  pilot.  Previous  memory  probes  have  required  little  more  than  recogni¬ 
tion  memory  for  locations  or  for  identity  of  a  stimulus  or  for  both  (e.g.,  the 
location  of  dials  and  the  identities  of  readings).  In  the  present  experiment, 
the  secondary  task  included  additional  processing  demands  for  the  pilot  to 
mentally  manipulate  and  to  transform  data  in  addition  to  storing  the  result  for 
later  retrieval.  It  was  assumed  that  the  increased  memory  loads  would  Interfere 
with  the  primary  task  in  a  linear  fashion. 

There  is  some  evidence  for  the  importance  of  the  training  criterion  level 
for  the  purely  cognitive  task  of  verbal  learning.  Cofer  (1969)  presented  some 
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of  Weitz's  arguments  on  the  importance  of  the  learning  criteria.  In  that  report, 
Weitz  (1961)  re-analyzed  the  data  from  a  paired-associate  study  which  was  con¬ 
ducted  by  Cramer  and  Cofer  (1960).  When  Weitz  re-analyzed  that  data,  he  used 
several  different  criteria  of  second  list  learning.  He  found  that  the  conclu¬ 
sions  which  could  be  reached  varied  greatly  as  a  function  of  the  particular 
learning  criterion. 

It  seems  reasonable  to  expect  that  various  training  criteria  might  also 
impact  on  learning  to  fly  a  simulator:  a  task  which  involves  both  psychomotor 
and  cognitive  components.  That  would  be  particularly  true  when  an  additional 
cognitive  task  was  demanded  of  the  subject.  Curiously,  we  could  not  locate  any 
studies  which  used  training  criteria  as  an  independent  variable  to  assess  the 
strength  of  such  effects.  Therefore,  the  purpose  of  this  study  was  to  compare 
the  effects  of  two  levels  of  training  criterion  against  two  types  of  heading  indi¬ 
cator  visual  displays  in  a  flight  simulator. 

Method 

Design 

The  design  was  a  2  x  2  x  (5)  mixed  factorial  with  repeated  measures  on  the 
last  factor.  There  were  two  levels  of  compass  heading  error  tolerance  (5  versus 
10  degrees)  and  there  were  two  types  of  compass  heading  error  indicators  (typi¬ 
cal  dial  compass  versus  light  display).  There  were  five  levels  of  memory  load 
(a  no-load  control,  and  four  loads  involving  increasing  amounts  of  information 
to  process) . 

Subjects 

The  subjects  were  male,  novice  Air  Force  Academy  cadet  pilots  who  had  com¬ 
pleted  at  least  nine  hours  in  a  T-41  aircraft.  The  median  was  21  hours.  The 
cadets  were  randomly  assigned,  seven  to  each  of  the  four  experimental  cells,  in 
order  of  their  appearance  for  the  experiment. 
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Flight  Simulator.  The  flight  simulator  was  the  Link  Group  -  General  Pre¬ 
cisions  Systems,  Incorporated,  General  Aviation  Trainer-1  (GAT-1),  Model  Number 
N1926P.  The  GAT-1  was  modified  with  black  curtains  to  preclude  any  visual  scene 
from  outside  the  cockpit.  The  GAT-1  approximates  the  flight  characteristics  of 
the  T-41C.  The  yoke  was  modified  to  include  a  button  on  its  top  left  side  which 
was  used  to  signal  responses  to  cognitive  task  questions. 

Peripheral  equipment  included  a  console  with  cockpit  repeater  instruments; 
an  intercom  system  to  provide  instructions;  and,  a  Brush  six-channel  pressure 
ink  recorder  to  record  heading,  airspeed,  vertical  velocity  and  responses  to  the 
secondary  cognitive  task. 

Heading  Indicators.  Two  groups  of  subjects  used  the  typical  T-41  dial 
compass  which  was  mounted  on  the  instrument  panel.  The  remaining  two  groups  used 
only  a  head-mounted  system  of  peripheral  vision  signals  with  the  dial  compass 
masked  from  view.  The  LED  signals  consisted  of  five  light-emitting  diodes  (LEDs), 
green  in  color,  wired  in  series.  Each  LED  was  2.4  mm  in  diameter  with  the  LEDs 
placed  six  mm  apart.  The  displays  provided  retinal  projections  at  55  degrees  of 
arc  subtended  from  the  front,  center  of  the  person’s  head.  Onset  of  the  light 
occurred  only  when  the  pilot  exceeded  the  desired  heading  by  ±  one  degree.  In 
order  to  correct  an  out-of-tolerance  condition,  the  pilot's  correct  response  was 
to  turn  towards  the  light  and  null  the  error. 

Closed-Circuit  Television.  The  visual  secondary  task  was  presented  via 
a  closed-circuit  television  monitor  positioned  in  the  GAT-1  windscreen. 

Each  visual  presentation  was  a  series  of  four  slides  and  a  no-slide  control 
condition  (see  Figure  1).  The  minimal  memory  load  target  slide  consisted  of 
only  a  series  of  six  ’’underlining"  dashes.  Low  memory  load  had  the  six  dashes, 
a  letter  above  one  dash  and  a  single  digit  number  above  a  different  dash. 

Moderate  memory  load  had  the  six  dashes,  one  letter,  and  different  single  digit 


numbers  above  three  of  the  dashes.  High  memory  load  was  comparable  except  that 
there  were  five  different  numbers  plus  the  letter. 
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Figure  1.  Target  Slides 

Three  probe  slides  (see  Figure  2)  followed  each  target  slide.  The  first 
probe  contained  the  six  dashes  and  an  arrow  pointing  down  towards  one  of  them. 
The  subject  had  to  respond  "yes"  or  "no"  by  pressing  the  button  on  the  yoke  to 
indicate  whether  or  not  the  arrow  pointed  to  the  location  of  the  letter  on  the 
target  slide.  The  second  probe  slide  contained  only  a  letter.  This  probe  asked 
whether  or  not  this  was  the  letter  on  the  target  slide,  regardless  of  its  loca¬ 
tion.  The  third  probe  slide  asked  whether  or  not  the  numerical  value  shown  was 
equal  to  the  transformation,  i.e.,  the  sum  of  the  digits  on  the  target  slide. 
Sixty  target  slides  were  presented,  15  at  each  level.  Each  slide  was  shown  for 
three  seconds  with  a  three  second  delay  between  slides. 

(1)  _  Jj _ 

(2)  E 

(3)  24 
Figure  2.  Probe  slides. 

Procedure 

Training.  Subjects  were  told  that  the  purpose  of  the  study  was  to  investi¬ 
gate  their  ability  to  respond  to  an  additional  task  while  flying  the  simulator. 
They  were  instructed  that  their  first  priority  was  to  fly  the  simulator  and  that 
the  extra  task  was  secondary. 


All  pilots  were  trained  to  criterion  on  a  vertical  S-"A"  aerial  maneuver 
for  one  hour.  This  maneuver  consists  of  ascending  to  an  altitude  of  2000  ft 
(656.17  m),  then  alternately,  ascending  and  descending  250  ft  (82.02  m)  above 
and  below  the  baseline.  This  maneuver  had  to  be  performed  while  holding  to  a 
heading  of  270  degrees,  airspeed  of  80  mph  (35.76  m/sec),  and  vertical  climb  and 
descent  at  500  ft  per  minute  (2.54  m/sec).  This  maneuver  is  very  demanding  and 
requires  frequent  motor  activity  and  instrument  scanning  and  cross-checking. 

For  two  of  the  experimental  groups,  the  criterion  consisted  of  performing 
the  vertical  S-"A"  without  deviating  more  than  10  degrees  from  the  desired  head¬ 
ing.  For  the  remaining  two  groups,  the  error  tolerance  was  five  degrees.  Thus, 
there  were  four  experimental  groups:  5-DIAL,  10-DIAL,  5-LED,  and  10-LED.  An 
instructor  pilot  monitored  performance  during  training  and  supplied  prompts 
whenever  heading  deviations  exceeded  either  the  five  or  10  degree  training  cri¬ 
terion. 

Experiment.  Experimental  sessions  occurred  on  the  second  day  following 
training.  Each  subject  was  allowed  five  minutes  of  practice  (two  trials)  after 
which  four  trials  of  the  vertical  S  were  performed.  The  subjects  all  performed 
a  secondary  task  involving  responding  to  60  sets  of  visual  stimuli  and  memory 
probes.  The  duration  of  each  stimulus  display  was  three  seconds  with  a  three 
second  inter-stimulus  delay. 

Each  visual  presentation  consisted  of  one  of  five  types  of  video  displays 
(see  Figure  1),  providing  five  levels  of  memory  load:  (1)  zero  or  no  load  con¬ 
trol,  (2)  memorizing  that  a  display  had  blanks,  (3)  memorizing  one  letter,  its 
location,  and  the  sum  of  one  number,  (4)  memorizing  one  letter,  its  location, 
and  the  sura  of  3  numbers,  and  (5)  memorizing  one  letter,  its  location,  and  the 
sura  of  5  numbers.  At  each  level  of  memory  load,  the  cadets  engaged  in  three 
cognitive  activities:  1)  localization,  2)  identity,  and  3)  transformation  (see 
Figure  2).  The  purpose  of  imposing  the  memory  loads  was  to  approximate  levels 
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of  cognitive  demands  analogous  to  the  information  processing  demands  of  scanning 
and  cross-checking  instrument  displays.  All  conditions  were  counterbalanced. 

Dependent  Measures.  The  following  dependent  measures  were  scored: 

(1)  Compass  Heading  errors.  These  errors  were  scored  as  absolute 
deviations  from  270  degrees,  sampled  every  five  seconds  during  each  two  minute 
trial. 

(2)  Compass  Heading  overcorrections.  These  errors  were  scored  as 
ratios  of  the  number  of  times  the  pilot  crossed  from  one  side  of  the  desired  head 
ing  (270°)  to  the  other  side  (e.g.,  268°  to  273°)  per  trial. 

(3)  Airspeed  errors.  These  errors  were  scored  as  absolute  deviations 
from  80  mph  (35.76  m/sec)  sampled  every  five  seconds. 

(4)  Rate  of  Ascent/descent  errors.  These  errors  were  scored  as 
absolute  deviations  from  the  required  500  ft/min  (2.54  m/sec)  sampled  every  five 
seconds  after  the  attitude  was  established. 

(5)  Error  Time.  This  score  consisted  of  the  proportion  of  time  the 
pilot  erred  by  exceeding  the  five  or  10  degree  criterion  during  the  experiment. 

(6)  Memory  load  scores.  These  scores  consisted  of  the  percent  of 
correct  responses  to  the  60  memory  probes  in  the  secondary  task. 

Results 

Table  1  presents  the  means  and  standard  deviations  for  all  measures  of  per¬ 
formance,  at  each  level  of  memory  load.  A  2  (heading  error  tolerance)  x  2  (com¬ 
pass  type)  x  (5)  (memory  load)  mixed  analysis  of  variance  on  these  data  yielded 
significance  for  the  memory  load  main  effect,  as  follows:  (a)  compass  heading 
errors,  £-8.25,  £«<. 01;  (b)  heading  overcorrections,  F*15.30,  £<C.01;  (c)  air¬ 
speed  errors,  JF-4.92,  £<C  .01  (all  df  =  4,96);  (d)  Error  Time,  £  (1,  15)  ■  8.26, 
.05;  and  memory  load  scores,  £(3,66)  =  14.19,  £<T. 01.  The  ascent/descent 
rate  was  not  significant.  Separate  2-tailed  t-tests  were  calculated  between  the 


control  condition  and  each  of  the  four  memory  load  levels,  and  between  the  load 
levels  only.  These  results  are  also  outlined  In  table  1. 

The  ANOVA  also  yielded  significance  for  several  important  interactions 
between  the  type  of  compass  heading  indicator  and  the  error  tolerance  training 
criterion  used  in  training.  These  results  are  presented  in  table  2. 

The  heading  type  by  criterion  level  interaction  was  significant  for  heading 
errors,  F  (1,24)  ■=  7.056,  £<C  .05  and  accounts  for  19%  of  the  total  variance. 
Figure  3  presents  a  graph  of  the  means.  Subjects  in  the  5°  criterion  groups  per¬ 
formed  better  only  if  they  were  in  the  normal  heading  indicator  group.  However, 
if  the  training  session  allowed  them  10°  heading  deviation,  they  subsequently 
performed  better  only  if  they  received  heading  information  from  the  peripheral 
lights.  _ 


5°  ]0° 

TRAINING  CRITERIA 


Figure  3.  Mean  absolute  heading  deviations 
as  a  function  of  heading  indicator  type  and 
difficulty  of  training  criteria. 

Figure  4  illustrates  the  significant  interaction  of  training  criterion 
with  heading  type  for  overcorrections,  _F  (1,  24)  -  11.649,  ]>  i^.Ol  and  accounts 
for  21%  of  the  variance.  In  this  case,  those  trained  to  the  5°  heading  deviations 
criterion  had  a  smaller  ratio  of  cross-overs  if  they  used  peripheral  lights 
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TABLE  1 


Means  and  Standard  Deviations  as  a  Function  of  Memory  Load  Level8 


Memory  Load  Levels 
Minimal  Low 


Measures 

Heading  Errors 
Overcorrections 
Airspeed  Errors 
Rate  Errors'5 
Error  Time 
Memory  Scores 


SD 

Control 

.11 

3.63d 

1.60 

45.72 

.17 

4.10 

3.74 

105.23 

1.55 

6.61 

1.33 

_c 

4.05 

4.17 

41.41® 

34.01 

4.36 

4.73 

107.12 

105.42 

9.41 

10.32 

95.58 

91.14 

Moderate 

High 

4.44 

4.38 

36.01 

29.84 

4.70 

5.07 

110.35 

100.89 

14.04 

11.83 

89.23 

83.48 

a.  All  tests  were  at  the  .01  level  except  Error  Time  (.05  level) 

b.  This  was  the  only  non-significant  main  effect. 

c.  There  was  no  score  in  the  control  condition. 

d.  Single  underline  indicates  significance  between  a  control  and 
other  levels. 

e.  Double  underline  indicates  significant  differences  between 
memory  levels. 

TABLE  2 

Summary  Results  on  Three  Dependent  Measures  for  the  Interaction  Between  Type  of 
Indicator  and  Level  of  Training  Criterion 


F 

Heading  Errors  7.06  (1 

Overcorrections  11.65  (1 

Rate  Error  7.08  (1 

8.26  (1 
2 

Average  Omega 


_E_ 


24) 

.05 

24) 

.01 

esi 

.05 

15) 

.05 

(Variance  Accounted  for)  -  21% 


19% 


21% 


17% 


Error  Time 


27% 


while  those  cadets  using  the  normal  heading  indicator  had  a  smaller  ratio  of 
cross-overs  at  the  10®  criterion-  — . 


5°  10° 


TRAINING  CRITERIA 

Figure  4.  Ratio  of  number  of  times  per  5  second  interval 
that  heading  crossed  over  the  desired  270°  as  a  function 
of  heading  indicator  type  and  level  of  training  criteria. 

The  significant  interaction  between  rate  errors  and  the  training  criteria, 

F.  (1,15)  •  8.26,  £  ^..05,  accounts  for  27%  of  the  variance  and  is  presented  in 
Figure  5.  At  the  5°  heading  deviation  criterion,  there  were  smaller  proportions 
of  rate  errors  if  the  cadets  used  the  normal  heading  indicator-  The  reverse 
was  true  at  the  10®  heading  deviation  criterion,  i.e.,  the  cadets  then  performed 
better  if  they  used  the  peripheral  lights  for  heading  information. 


5®  10® 


TRAINING  CRITERIA 

Figure  5.  Proportion  of  time  that  heading  equals  or 
exceeded  the  maximum  allowed  deviation  as  a  function 
of  the  training  criteria. 
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There  were  two  measures  of  flying  proficiency  that  were  not  related  to 
heading  information.  Airspeed  deviations  did  not  result  in  a  significant  statls* 
tic  for  the  heading  type  by  criterion  level  interaction.  The  heading  type  by 
criterion  interaction  was  significant  for  vertical  velocity  deviations,  I?  (1,24) 
■  7.080,  £  C.  .05  and  accounts  for  172  of  the  variance  (see  Figure  6).  Those 
trained  at  the  5°  criterion  performed  better  if  they  used  the  normal  heading 
indicator:  those  trained  at  the  10°  criterion  performed  better  if  they  used 
the  peripheral  lights. 


□  Normal  Indicator 
Q  Peripheral  Lights 


Figure  6.  Mean  vertical  velocity  deviations 
as  a  function  of  heading  indicator  type  and 
difficulty  of  training  criteria. 

DISCUSSION 

In  addition  to  flying  the  simulator,  subjects  were  required  to  respond  to 
a  secondary,  cognitive  task.  The  ability  to  respond  correctly  to  the  secondary 
task  decreased  with  Increasing  task  level  difficulty.  With  the  exception  of 
vertical  velocity  deviations,  simulator  performance  also  systematically  declined 
as  the  secondary  task  level  increased.  Although  pilots  were  told  that  their 
primary  responsibility  was  to  fly  the  simulator,  it  appears  they  were  unable  to 
isolate  the  secondary  cognitive  task  from  the  primary  task  of  flying  the 
simulator. 
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Contrary  to  the  original  predictions,  there  were  no  statistically  signifi¬ 
cant  main  effect  differences  for  either  the  type  of  heading  indicator  or  the 
training  criteria  levels  for  any  of  the  measures.  However,  those  two  factors 
did  have  statistically  significant  interactions.  Performance  on  the  simulator 
was  affected  by  the  interaction  between  the  type  of  heading  indicator  and  the 
training  criteria  levels.  Those  interactions  accounted  for  an  average  21Z  of 
the  variance  and,  therefore,  the  results  seem  to  have  practical  as  well  as 
statistical  significance. 

These  results  clearly  show  that  the  superior  simulator  performance  of  the 
5-DIAL  and  10-LED  groups  cannot  be  attributed  solely  to  the  type  of  heading 
indicator  used  nor  to  the  difficulty  level  of  the  training  criterion  alone. 
Instead,  their  performance  was  the  result  of  the  unique  contribution  of  type  of 
information  display  and  the  training  criterion  used  with  it.  The  concommltant 
failure  to  produce  significant  interactions  as  a  result  of  the  demands  of  the 
memory  load  tasks  seems  to  once  again  point  out  the  adaptability  of  the  pilot 
to  changing  workloads.  However,  this  adaptability  did  not  generalize  to  the 
primary  task  as  well  for  the  two  other  experimental  groups,  indicating  real 
differences  in  display  type  by  criterion  combinations.  Also,  contrary  to  our 
expectations  and  previous  results,  the  greatest  memory  load  proved  not  to  be 
the  most  decremental  on  primary  task  performance.  The  expected  linear  function 
was  not  found.  Presumably,  once  subjects  have  sufficient  memory  load  they 
engage  in  some  form  of  chunking  strategies.  The  memory  load  thereby  is  reduced 
to  fewer  items,  allowing  more  residual  attention  for  the  primary  task. 

These  Interpretations  are  strengthened  by  the  fact  that  five  of  the  six 
dependent  measures  are  convergently  consistent.  For  example,  both  the  5-DIAL 
and  10-LED  groups  produced  the  least  number  of  heading  errors,  and,  as  would  be 
expected,  the  greatest  number  of  heading  overcorrections.  This  indicates  these 
two  groups  were  not  only  the  most  accurate  in  maintaining  heading  but  they  were 
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also  the  most  responsive  In  their  attempts  to  be  accurate.  This  view  Is  further 
supported  by  the  fact  both  groups  also  produced  the  lowest  error  time  scores 
beyond  the  5  and  10  degree  training  criterion  thresholds.  Furthermore,  they 
also  produced  the  most  accurate  performance  in  ascent /descent  rate.  That  is,  a 
performance  measure  which  is  strongly  related  to  the  heading  correction  task 
because.  In  attempting  to  maintain  attention  to  heading,  the  pilot  may  sacrifice 
accuracy  in  his  vertical  climb  or  descent. 

Perhaps  some  of  the  divergent  results  from  earlier  flight  simulation  studies 
were  due  to  using  various  training  criteria.  For  example,  Pulg  et  al  (1978)  re¬ 
viewed  the  "motion  issue"  in  flight  simulation.  They  found  little  consistency 
relative  to  the  effectiveness  of  platform  motion  in  flight  simulation  research. 
The  results  of  this  research  indicate  that  inclusion  of  the  training  criteria  as 
an  independent  variable  might  aid  in  systematically  demonstrating  when  platform 
motion  is  beneficial. 

Previous  literature  suggests  that  the  choice  of  criterion  level  can  influ¬ 
ence  a  purely  cognitive  task.  This  study  indicates  that  the  choice  of  criterion 
level  also  impacts  research  which  also  has  psychomotor  components. 
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